Abstract
Agent-based modelling (ABM) is one of the most applied computational modelling research techniques in archaeology, because it is well suited to the questions asked by archaeologists. However, the access to vocational training for ABM is limited. In this paper we present the Open Educational Resources (OER) that we produced in an international cooperation. For this purpose we created online and open tutorials for ABM for archaeologists. In addition, how-to guides and support material was created, both for teachers and students. The material adheres to international (European) standards in relation to education and the Digital Skills Passport for archaeology. The teaching material is well tested using a diverse group of archaeologists during various international conferences and meetings. It was shown that the knowledge and skills of the participants in relation to ABM improved. In addition, the demand for the teaching material and the workshops was high, ensuring that the tutorials on ABM will be used in the future.
Computational modelling has become a standard research technique across most scientific disciplines, including archaeology. Among the many computational modelling techniques, the discipline has found agent-based modelling to be particularly well suited to the questions asked by archaeologists. Agent-based modelling is a simulation technique where autonomous agents interact with each other and with their environment. Their aggregated behaviour gives rise to system-level outcomes, thus enabling researchers to understand how micro (individual) mechanisms lead to macro (system-level) patterns.
While agent-based modelling importance in archaeological practice is steadily growing (Romanowska and Scherjon 2023) there are significant barriers to its wider adoption. One of the problems is the access to vocational training required to gain the necessary computer programming and modelling skills (Daems 2020). A survey of practitioners (Davies and Romanowska 2018) found significant deficiencies in the provision of ABM training; 76% of modellers were primarily self-taught with some degree of secondary support, i.e. 27% relied on peer support, only 7% had any sort of formal training at a workshop or summer school, while only 14% received training as part of a degree.
A consortium of Landward Research Teoranta, Aarhus University, Leiden University and Saxion University of Applied Sciences recognised this need, and sought to address the limited opportunities that archaeologists have to access high quality training. A project was set up to create and make available open learning materials that could be integrated into a wide range of training programmes, such as face-to-face training, employer-focussed continuing professional development seminars, webinars, MOOCs (Massive Open Online Courses) and self-directed learning.
This project, Agent-Based Modelling for Archaeologists, was delivered with the support of the Erasmus+ programme of the European Union.
The project sought to create Open (Open Access and Open Source) Educational Resources (OERs) for training in Agent-Based Modelling for Archaeologists. These OERs were to be HTML and JavaScript based so they could be used on any device with a browser, even if it is not connected to the internet. The full programme consists of five hands-on practical tutorials, taking learners through the process of creating the digital models. The modular architecture ensures that they can be independent or incorporated into other teaching tools like learning management systems. They are also deliberately ‘method agnostic’ so they can be incorporated into any tools from self-teaching to MOOCs, and are supported by a set of support materials to encourage use of the OERs.
The activities the project implemented were:
development of a hands-on vocational training programme, aligned with the recently published textbook Agent-Based Modelling for Archaeology: Simulating the Complexity of Societies (Romanowska, Wren and Crabtree 2021);
conversion of the training programme into modular learning resources that are interactive, based on HTML and JavaScript standards;
creation of support materials to encourage use of the OERs;
creation of how-to guides to help trainers incorporate OERs into their teaching;
creation of a code and learning materials repository that will allow the project to continue sustainability after the European Union support has ended and to facilitate the creation of an open-source community to manage and improve the OERs in the future;
development of the framework for skills acquisition aligned with the Digital Skills Passport;
promotion through a series of multiplier events.
The expected results and outcomes, for the target audience of professional archaeologists, were:
a surge in professionals who can build, apply and critique archaeological simulation thus opening new research avenues for the discipline as a whole;
increased level of digital competence, specifically in advanced computer skills that are highly transferable to other industries. This includes technical skills such as programming but also “soft” skills related to computational thinking;
improved opportunities for archaeologists to work in other European countries (transnational mobility) through matching the skills being delivered with the European Commission’s Digital Competence Framework for Citizens (2.2) and the sectoral Skills Passport for Archaeology, which means that it will be easier for individual workers to demonstrate their skills to employers across Europe;
increased opportunities for professional development by having access to professional training that can be incorporated into formal or non-formal/informal learning.
For the secondary target audience of vocational training providers in archaeology e.g. employers, colleges and not-for-profit organisations, the expected results and outcomes were:
increased quality of training in Agent-based Modelling that is free and open;
access to teaching materials for skills that are in high demand and aligned with employers’ needs.
We were able to draw on previous experiences with a small private online course (SPOC) on ‘Modelling and Simulation in Archaeology’ that some of us designed for graduate teaching at the Faculty of Archaeology, Leiden University, where we taught this course in 2016/17 and again in 2018/19 (Scherjon, Romanowska and Lambers 2019). For this course we developed new online teaching materials comprising pre-recorded lectures, practical exercises, reading assignments and exams. While we greatly benefited from that experience, the target group of the SPOC was graduate students with prior experience in digital archaeology, whereas the ERASMUS+ project was aimed at a much broader target group of archaeology students and professionals whose computational literacy could not be assumed.
The project team developed five online tutorials. The tutorials were first designed using storyboards and later converted into interactive guided walk-throughs using NetLogo Web (https://www.netlogoweb.org/launch) and Javascript and hosted on GitHub (https://github.com/ABMArchaeologists/ABMA_tutorials). During the project, initial testing was done by all authors and the issues functionality in GitHub was used to tackle bugs.
The ABMA learning materials assume that participants have learning skills at least at a secondary education level. They are suitable for participants with at least some background in archaeology, such as those typical among students after the first year of a Bachelor study in Archaeology, i.e., above EQF levels 4 or 5. In addition, the tutorials are aimed at professionals working in archaeology who have some experience with computer applications. We also assume that learners are proficient in English and have at least the Reading B1 level, but Reading B2 is recommended (Council of Europe 2020). The tutorials were also developed with a secondary aim to develop learners’ general skills in the following competence areas: 1. Information and data literacy, 2. Communication and collaboration, 3. Digital content creation and, 5. Problem solving (European Commission, Joint Research Centre et al. 2022).
After initial debugging by the project team, the tutorials were tested on a wider sample of participants during several online and in-person events all of which were run by members of the project team (Table 1). Recognising the difficulty of delivering practical training in an online environment (Microsoft Teams and Zoom) during the online events several members of the team were present which enabled participants to go to break-out rooms with an expert if they needed additional help or wanted to discuss things. Throughout the project we sought feedback from the participants and used it to improve the tutorials prior to the following events. The feedback was obtained through two surveys using Qualtrics (https://www.qualtrics.com/). The participants of the events were asked to respond to one survey before the start of the event and one after the end of the event. The first survey focused on the background of the participants and their original level of knowledge in relation to ABM (see appendix 1 for the questions). At the end of the event the participants were asked to answer a second survey. Questions of this survey were aimed at measuring the effectiveness of the tutorials and getting feedback on the workshop and tutorials (see appendix 2 for the questions). For some events participants had to register beforehand and we were able to send the pre-workshop survey by email. At other events no registration was possible or necessary and the pre-workshop survey was shared at the start of the event. The post-workshop survey was distributed using QR-codes or links at the end of the event.
| Conference or event | Period | Online, in person or hybrid | Number of participants | Number of registrations |
|---|---|---|---|---|
| CAA Amsterdam | April 2023 | In person | 36 | 53 |
| EAA Belfast | August 2023 | In person | 21 | No registration |
| CAA-DE/NL-Fl Online workshop | October 2023 | Online | 40 | 76 |
| Reuvensdagen | November 2023 | In person | 13 | No registration |
| CAA-UK | November 2023 | Hybrid | 53 | No registration |
| Leiden (course) | December 2023 | In person | 11 | No registration |
| Aarhus | January/February 2024 | Online | 176 | >500 |
| Saxion | March 2023-January 2024 | In person | 18 | No registration |
| Total | 368 | >628 |
The surveys were analysed using R (R Core Team 2023). The following packages were used for analyses: ggplot2 (Wickham 2016), dplyr (Wickham et al. 2023), tidyr (Wickham, Vaughan and Girlich 2023), forcats(Wickham 2023a), lubridate (Grolemund and Wickham 2011) and stringr (Wickham 2023b). For the sake of reproducibility (Marwick 2017) this paper was written in R Markdown (https://rmarkdown.rstudio.com/), with data and code available at https://github.com/ABMArchaeologists/ABMA_paper (Visser et al. 2024b).
Over the course of the project three groups of students worked on the project during the Smart Solutions Semester at Saxion University of Applied Sciences. This is an interdisciplinary semester in which students of at least three different study-programmes or disciplines work together on a complex problem/project (https://www.saxion.edu/business-and-research/collaborate-with-saxion/smart-solutions). Within the semester students have to come up with new ideas or solutions to help a client with a wicked, or at least challenging, problem. The backgrounds of the various students we worked with were diverse: Applied Computer Science, Archaeology, Business Management Studies, Creative Business, Creative Media & Game Technologies, and ICT. The aim of involving interdisciplinary groups in the project was to involve students in current research projects, to enable them to learn new things and share their innovative ideas and solutions with us. The students groups were supplementary to the deliverables of the project, but their new ideas were beneficial to the project. Their help during workshops was useful, but they also were engaged in developing educational material (Aalpoel et al. 2024), testing the tutorials, developing a style for the website (Jutte et al. 2024) and other materials.
The ABMA tutorials are publicly hosted on GitHub (https://github.com/ABMArchaeologists/ABMA_tutorials/) and are citeable (Rocks-Macqueen et al. 2024). Anyone interested can download the tutorials and their source code. To increase the accessibility and visibility of the tutorials, they are also hosted on a static website (https://abmarchaeologists.github.io/ABMA_website) and can be followed without the need to download any software. The source code of the website is also openly available (Jutte et al. 2024).
Besides, the tutorial, the website provides: a link that can be used for feedback, basic explanations of agent-based modelling, an introductory presentation from one of the workshops, videos from an online course at Leiden University (Scherjon, Romanowska and Lambers 2019) and external resources that people can use to learn agent-based modelling. The website was mainly developed by a student group from Saxion University of Applied Sciences (Jutte et al. 2024).
The final set of ABMA tutorials is available for download at https://github.com/ABMArchaeologists/ABMA_tutorials and is citeable (Rocks-Macqueen et al. 2024). The set consists of the following tutorials:
Tutorial 1: Introduction to ABM
Tutorial 2: Beginning with NetLogo
Tutorial 3: Expanded ABM skills
Tutorial 4: Intermediate ABM
Tutorial 5: How to Model
Each tutorials consists of a number of individual lessons, which guide the learner in a self-paced manner through an increasingly challenging set of exercises. Collectively, they build the skills and knowledge necessary to become a confident agent-based modeller.
In the first tutorial the users learn what simulations and agent-based models are and how they can help in archaeological research (Figure 1). This tutorial consists of four lessons and makes use of the Artificial Anasazi model (Axtell et al. 2002). The first two lessons introduce the learner to simulation in general and Agent-Based Modelling in specific. Various concepts related to ABM, such as the definitions of model and simulation, are introduced and the participants take their first steps in NetLogo syntax. The third lesson explains how ABMs are used in archaeological research. The final lesson introduces the learner to the NetLogo Interface and the difference between the Interactive and Authoring mode. This tutorial builds a foundation for working in NetLogo and with ABM. All activity stays within the entry level of difficulty, although some higher level concepts are lightly touched upon.
Figure 1: Screenshot of lesson 3 in tutorial 1 showing the population dynamics of the Long House Valley in the Artificial Anasazi model created by Axtell et al. (2002).
In tutorial 2 the user will learn the basics of NetLogo by making their simulation on the Out of Africal dispersal of Homo sapiens (Young and Bettinger 1995). The learner works through the basics of NetLogo syntax and learns how to set up a simulation and visualize its outcomes. This tutorial consists of nine lessons. This tutorial intends to guide the learner from the introduction to the beginner level of proficiency. The learning curve is relatively gentle. This is achieved by working with the NetLogo web interface that enables highlighting specific words and buttons and giving visual cues as to how to proceed. The tutorial introduces several commonly used built-in functions in Netlogo (called primitives) and introduces the general structure of all models consisting of the initialization phase and the main simulation loop. It walks the participants through the most important features of the NetLogo world, such as dimensions, coordinates and the origin point of the grid on which the simulation runs. The learners gain their first stripes in programming by using simple loops and functions as well as conditional statements. The next level is achieved by introducing the topic of function definition and different types of variables. Exporting data produced by the model through plots is also explained.
In the third tutorial, the learner builds a simple trade model, again based on a published simulation (Romanowska 2018). Here learners expand their NetLogo programming skills with more complex syntactic structures as loops, lists and reporters. They are introduced to some key computational techniques like modular code development and debugging which become important alongside this increased coding complexity. This tutorial consists of seven lessons. It intends to guide the learner from the beginner to the intermediate level of proficiency. This is achieved by introducing standards of code development such as modular code, pseudocode, debugging and annotating the code. In addition, the tutorial introduces custom agent breeds, visualization with labels and reporters, plots and monitors. This takes the learner to a more advanced level, since it develops knowledge of the programming language combined with diagnosing unexpected behaviour and problem solving.
In tutorial 4 the learner works with the seminal Sugarscape model (Epstein and Axtell 1996) to further expand their agent-based modelling and NetLogo skills. The learner explores how to set up more complex interactions between agents and the environment. Furthermore, the principles of setting up good experiments and validating models is explained. This tutorial consists of eight lessons. It intends to solidify the learner’s understanding at the intermediate level of proficiency by introducing yet more complex agent-environment interaction, alternative ways to visualize the environment and modelling spatial dynamics. Creating toy landscapes is central to lesson 5. The learner dives into the topic of experiment design at a practical level but also with regards to the scientific principles, such as validation. Thus, they are guided through the process of setting up experiments in NetLogo and collecting results using monitors and plots. The topic of validation of agent-based models is critical for most archaeological applications and thus several ways to compare simulation results with the archaeological record are explored.
In the final and fifth tutorial the learner learns more about how to incorporate agent-based modelling in archaeological research. This tutorial focuses less on programming in NetLogo and more on the principle and standards of computer-based modelling. The learner learns about the different phases of the model development process. This tutorial consists of five lessons. It summarizes the material delivered in the previous lessons into one coherent framework consistent with an intermediate level of proficiency. This tutorial approaches more theoretical aspects in a practical environment. The learner revisits all phases of the model development process starting with the conceptual phase and finishing with the analysis and interpretation of results. There is a particular focus on discussing what kind of research questions are suitable for modelling and how to pick the right modelling technique. The importance of properly conceptualizing a model before starting the technical phase of the model development is highlighted as it is a common issue for less experienced modellers. The experiment design phase, including parametrisation, validation and the analysis and interpretation of the models is explained at length since it often proves challenging for students. On a practical side, the BehaviorSpace - NetLogo’s experiment environment - is explained to enable the learners to batch run their models. Finally the dissemination phase of the model development is also included to ensure that the standards in model publication are widely known and the learners explore the fundamental importance of replication for advancing scientific knowledge.
The tutorials were tested during various workshops and updated afterward. As mentioned earlier, the participants of these workshops were asked to fill in a questionnaire both before and after the workshop.
A large proportion of the participants of the workshops gave us information using the survey before the workshops (172 of 368 participants). The respondents came from at least 40 countries that represented almost all continents (see Figure 2). There was a clear skew towards participants from Europe. The number of female respondents slightly outnumbered the male ones and a small group did not share their gender, while two identified as non-binary (see Figure 3). The ages of the respondents ranged from below 20 to over 70, although the majority fell between 20 and 40 years old indicating that the audience was predominantly composed of early career archaeologists. It seems that the different events also had a slightly different distribution of both gender and age. For example, more older people attended the workshop at the CAA conference in April 2023 and more men were present during the workshop at the Reuvensdagen in November 2023. This might be due to the differences of audiences at the conferences. For the Reuvensdagen the number of participants of the workshop was relatively low and this variance might be due to chance.
Figure 2: The nationality of the participants that filled in the survey.
Figure 3: The gender and age distribution of the respondents for each workshop.
Most of the respondents reported having a level of computer literacy already including particular computational skills common in archaeology, for example working with word processors, GIS and spreadsheets (see Figure 4), and 111 of the respondents had some prior knowledge of ABM, while 59 were completely new to the subject. The majority (158) had never applied ABM to their research before participating in the workshops with only 14 participants who had experience of developing archaeological ABM. It is interesting to note that many respondents (108) did know what kind of software was available for ABM (see Figure 5).
Figure 4: The computer skills of the respondents.
Figure 5: Respondents knowledge of ABM per event (top), and knowledge of the various softwares for ABM (bottom), before participating in the workshops.
As shown above, the respondents had some knowledge on ABM in general, but did not know how to apply it or had never applied it before. The respondents were also asked how they rated the available resources on ABM (Figure 6). While 64 of them had no opinion on the subject, a proportion of the respondents (28) answered that they rated the available theory on ABM as limited and only a small group (39) as sufficient or better. This clearly shows the need for more and better educational material.
Figure 6: Respondents opinion on the quality of theory on ABM faceted out by event.
During the workshops the participants worked in a self-paced manner, often on their own, but sometimes working together and discussing the tutorials (Figure 7). Some participants engaged in discussion with the teachers to learn how they could apply ABM in their research of discuss possibilities for the application of ABM in general. Individual progress varied depending on skills and preferences of the participants. While bugs were still a problem in earlier workshops, participant-teacher interaction was more focused on content-related learning problems in later installments.